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STEGANOGRAPHIC METHOD AND DEVICE 

Definitiong 

5 Several tarns of art appear frequently in the following. For ease of refo-ence they 
are defined here as follows: 

"Content" refers to multimedia content. This term encompasses the various types of 
faformation to be processed in a multimedia entertainment system. Content 
10 specifically refers to digitzed audio, video or stiU unages m the contract of this 
discussion. This information may be contained within files on a multimedia 
computer system, the files having a particular format specific to the modality of the 
content (sound, images, moving pictures) of the type of systems, computer or 
othowise, used to process the content. 

IS 

•Digitized" refers to content composed of discrete digital samples of an otherwise 
analog media, which approximate that media inside a computer or other digital 
device. For instance, the sound of music occurs naturally, and is experienced by 
humans as an analog (continuous) sound wave. The sound can be digitized into a 
20 stream ofdiscrete samples, or numbers, each of which represents an approximate 
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vahie of the amplitude of the real analog wave at a particular instant in time. These 
samples can be stored in files m a computer and then used to recreate the nri ginni 
sound wave to a hi^ degree of accura^. 

In genoal. content entering a digital system is digitized by Analog to Digital 
conwters (A/D) and analog media are recreated by the digital system using a 
Digital to Analog (D/A) converter. In the context of this discussion content is 
always digitized content. 

"Glyptography" is a field covering numerous techniques for sctambling information 
conveymgmessages so that when the message is conveyed betwem the sendo- and 
recttver an unintended party v/bo intercq)ts this message cannot read it, or extract 
usefiil information fiom it 



A "Public Key Cryptosystem" is a particular cryptographic system where all parties 
15 possess pan? of keys for enoyption and decryption. Parties to this type of qrstan 
fi:eely distribute their public keys, which other may use to encrypt messages to the 
owner of the public key. Such messages are decrypted by the recover with the 
private key. Private keys are never distributed. A message encrypted with a public 
key can only be decrypted with the corresponduig private key, and vice versa. A 
20 message encrypted with a private key is said to have been signed by the owner of 
that key. Anyone in possession of the public key may decrypt the message and 
know that it was encrypted, and thus signed, by the owner of the pubUc key, since 
only thqr possess the corresponding private key. 

25 "Sinography" is a field distinguished from cryptography, but associated with it, 
that covers numerous methods for hiding an informational message within some 
other medium, perh^s another unrelated message, in such a manner that an 
unintended party who mtercepts the medium carrying the hidden message does not 
know it contains this hidden message and therefore does not obtain the information 

30 in the hidden message. In other words, steganography seeks to hide messages in 
plain view. 
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Background of the Invention 

Id the curroit environment of computet networics and the proliferation of digital or 
5 digitized muhimedia content whidi nu^ be distributed ova- such netw(»ks, a key 
issue is copyright protection. Copyright protection is the ability to prevent or deter 
the proliferation of unauthorized copies of copyrighted works. It provides a 
reasonable guarantee that the author of a copyrighted work will be paid for each 
copy of that work. 

10 

A fundamental problem in the digital world, as opposed to the worid of phyacal 
media, is that a unlunited number of pofect copies may be made from any piece of 
digital or digitized content. A pofect copy means that if the ori^nal is comprised of 
a ©ven stream of numbers, then the copy matches the original, exactly, for each 
15 numb^ in the stream. Thus, there is no degradation of the original signal during the 
copy operation. In an analog copy, random noise is always introduced, degrading 
the copied agnal. 

The act of maldng unlicensed copies of some content, digital or analog, wh^er 
20 audio, video, software or other, is generally known as piracy. Piracy has been 

committed for the purpose of «th«- profit fi-ora the sale of sudi unlicensed copies, 
or to procure for the "pirate" a copy of the content for personal use without having 
paid for it. 

25 The problem of piracy has be«i made much worse for any type of content by the 
digitization of content. Once content enters the digital domain, an unlimited number 
of copies may be made without any degradation, if a pirate finds a way to break 
whatever protection scheme was established to guard against such abuses, if any. 
In the analog worid, there is goieraUy a degradation in the content (signal) with 

30 each successive copy, imposing a sort of natural limit on volume of piracy. 
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To date, three gen^ types of schemes have been implemented in an attempt to 
protect copyrights. 

l)Enayption 
5 2) Copy Protection 

3) Content Extensions 

Copy Protection and Content Extensions generally apply in the digital world only, 
while a scheme related to Encryption, commonly known as scrambling, my be 
10 applied to an analog signal. This is typical in analog cable qrstems. 

Enciyption scrambles the content. Before the content is made ready for ddiveiy, 
v^ether on floppy disk, or over a netwoiic, it must be encrypted, or soambled. 
Once the content has been enciypted, it cannot be used until it is decrypted, or 
15 unscrand)led. Enoypted audio data might sound like incomprehensible screeching, 
-while an enciypted picture or video might appear as random patterns on a screen. 
The prindple of enciyption is that you are free to make as many copies as you want, 
but you can't read anything that makes sense until you use a special key to deciypt, 
and you can only obtun the key by paying for the content. 

20 

Enciyption has two problems, however. 1) Pirates have historically found ways to 
cradc oiciyption, in effect, obtaining the key without having p^d for it; and 2) 
Once a single le^timate copy of some content has been decrypted, a pinrte is now 
free to make unlimited copies of the deciypted copy. In effect, in order to sell an 
25 unlimited quantity of an enciypted piece of software, the pirate could simply buy 
one copy, which they are entitled to deciypt. 



Copy Protection includes various methods by which a software engaieer can write 

the software in a clever manner to determine if it has been copied, and if so to 
30 deactivate itself Also included are undocumented changes to the storage foimat of 
the content. Copy protection was generally abandoned by the software industiy. 
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^ce pirates were general^ just as clever as the software engines and figured out 
ways to modify thdr software and deactivate the protection. The cost of developii^ 
such protection was not justified considering the level of piracy which occurred 
despite the copy protection. 

5 

Content Extension refers to any system which attaches some extra information to 
the original content vAAch indicates whether or not a copy may be made. A 
software or hardware system must be spedfically built around this scheme to 
recognize the additional mformation and interpret it in an appropriate manner. An 
10 example of such a systm is the Serial Copyright Management System embedded in 
Distal Audio Tape (DAT) hardware. Under this system, additional information is 
stored on the disc immediately preceding each track of audio content which 
indicates whether or not it can be copied. The hardware reads this information and 
uses it accordingly. 

15 

A fimdamental problem with Encryption and Content Extension is the "rogue 
engineer". An employee who helped design such a system or an individual with the 
knowledge and means to analyze such a system can modify it to ignore the 
copyright information altogether, and make unlicensed copies of the content. Cable 
20 piracy is quite common, ^ded by illicit decoder defaces built by those who 

undo-stand the technical details of the cable encryption system. Although the cable 
systems in question were actually based on analog RF signals, the same principle 
applies to digital systems. 

25 The practical considerations of weak enciyption schemes and rogue engineers have 
served to lunit the faith which may be put in such copyright protection schemes. 
The mvention disclosed herdn serves to address these problems with conventional 
systems for digital distribution. It provides a way to enforce copyright online. The 
invention draws on techniques from two fields, cryptography, the art of scrambling 

30 messages so that only the intended recipient may read them, and steganography, a 
tenn applied to various techniques for obscuring messages so that only the intended 
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parties to a message even know that a message has been sent, thus it is tenned 
herdn as a st^-dpho-. The ste@a-dpher is so named because it uses the 
steganographic technique of hiduig a message in multimedia content, in combmation 
with multiple keys, a concept originating in ciyptcgraphy. However, instead of 
5 using the keys to encrypt the content, the stega-cipher uses these keys to locate the 
hiddra message within the content. The message itself is enciypted which serves to 
further protect the message, verify the validity of the message, and redistribute the 
information in a random manner so that anyone attempting to locate the message 
without the keys cannot rely on pre-supposed knowledge of the message contents 
10 as a hdp m locating it 

Summary of the Invention 



The invention disclosed herdn combines two techniques, steganography - obscuring 
15 information that is otiierwise in plam sight, and cryptography - scrambling 

information that must be sent over unsecured means, in a manner such that only the 
intended recipient may successfully unscramble it. The net effect of tiiis system is to 
specifically watermark a piece of content so tiiat if it is copied, it is possible to 
determine who owned the original firom which the copies were made, and hoice 
20 determine responsibility for the copies. It is also a feature of the system to uniquely 
identify the content to which it is applied. 

For a comprehensive discussion of cryptography, its tiieoiy, applications and 
specific algoritimis, see APPLIED CRYPTOGRAPPIY, by Bruce Schneier, which is 
25 herdn incorporated by reference at pages 66-68, 387-392. 

Steganography is discussed briefly in THE CODE BREAKERS by David Kahn, 
which is herein incorporated by reference at pages xiii, 81-83, 522-526, and 873. 
An example appUcation, Stego by Romana Machado, is also available for the Apple 
30 Macintosh. Stego can be found at tiie uitemet uniform resource locator "ftpy/sumex- 
aiIllJtanfo^deduAlfo^nacfcmI«/sl^l^ This application demonstrates a simple 



steganographic technique to encode a text message into a graphical image without 
sigmficantly distoiting the ima^. 

The inveation improves upon the prior art by providing a manner for protecting 
copyright in the digital domain, which neither steganography or cryptography does. 
It improves specifically on steganography by making use of special keys which 
dictate exactly where within a larger chunk of content a message is to be hidden, 
and makes the task of extracting such a message without the proper key the 
equivalent of looking for a needle in a haystack. 



10 



The infonnation encoded by the St^-Cipher process serves as a watermaric which 
identifies individual copies of content l^ly licensed to specific parties. It is 
int^ with the content. It cannot be removed by omission in a transmissioa It 
does not add any overiiead to signal transmission or storage. It does allow the 
15 content to be stored to and used with traditional offline analog and digital media, 
without modification or significant signal degradation. These aspects of the stega- 
dpher all represent improvements to the art. That is, its forces would - be puates 
to damage the content in order to guarantee the disabling of the watennaric 

20 The invention described herein is used for protecting and enforcing copyrights in 
the digital or on-line domain, where there are no physical limitations on copying 
copyrighted contoit. 

The invention uniquely identifies every copy of multimedia content made using the 
25 mvention, composed of digitized samples whether compressed or uncompressed, 
inchiding but not limited to still digital images, digital audio, and digital video. 



30 



The invention is for use in meterware or pay-by-use systems where an online user 
incurs a chaise each time they access a particular piece of cortent, or uses a 
software title. 
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The invoitioii is for use as a genwal improv^oit to ctyptogi^hic techniques to 
increase the con^lexity of oyptanalysis on a given dpher. 

It is considered that the method and steps of the present invention wiU be modified 
5 to account for the efifectsofloss compression schemes on the samples and 

particulariy includes modification to handle MPEG compressed audio and video. 

It is considered that statistical data spreading and recovery techniques, error coding 
or spread spectrum processing techniques might be applied in the invention to 
10 handle the effects of loss compression, or counter the effects of a randomization 
attack. 

It is considoed that the apparatus desoibed might be fiirtho- spedalized and 
optimized in hardware by r^lacing general purpose data buses and CPU or DSP 
15 driven operations with hardwired circuitiy, incoiporated in one or more special 
purpose ICs. 

It is considered that the apparatus vwll be modeled and implemented in software on 
general purpose computer platforms. 

20 

It is considered that st^a-dpher hardware could be embedded in a consumer 
electronics device and used to not only identify content and copyright, but to enable 
use of that content 

25 DetaUed Description 

I Digital Copyright Stega-Cipher Protocol and the Decode/Encode 
Program 

30 The purpose of the program described here is to watermark digital multimedia 
content for distribution to consume? through onluie services in such a way as to 
meet the following criteria 
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Given a unique piece of multimedia content, composed of digitized san^>les, h is 
dearable to: 

1) Uniquely identify this particular piece of content from others in a manner which 
5 is secure and undeniable (e.g. to know whether a digital audio recording is "My 
Way" by Frank Sinatra, or "Stairway to Heaven", by Led Zeppelin), and in a 
manner such that this identification can be performed automatically by an electronic 
device or mechanism. 

10 2) Uniquely identify the copyright owner of the content, and the terms under vdiich 
it may be distributed in general, in a manner which is secure and undeniable. 

3) At such time as is necessaiy, additionally, uniquely identify in a secure and 
undeniable manner the licensed publi^er who received a particular copy of the 
IS content, and the terms under which they may redistribute or resell it. 



4) At such time as is necessary, additionally, uniquely identify in a secure and 
undeniable manner, the licensed subscriber who received a particular copy of the 
content from the publisher described in item 3. 

20 

The program described in more det^l below combines the techniques of 
ciyptography and steganography to hide a securely encrypted digital copyright 
certificate "wbidi contains uiformation satisfying the criteria listed above, in such a 
manner as to be integral with the content, like a watermark on paper, so that 

25 possession of the content dictates possession of the watermark information. In 
addition, the watermark cannot be "found" or successfully decoded, without 
possession of the correct "masks" or keys, available only to those legitimately 
authorized, namely, those parties to a commercial transaction involving the sale of a 
copy of the content. Fmally, the ability to distribute such watermarked content in a 

30 system which implements the watermark scheme is denied without a successfiilly 
decoded watermark. Because well known and tested ciyptogrj^hic techniques are 
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used to protect the certificate itsdC these Certificates are virt^ 

fiaiBC. FinaUy. the watennark cannot be erased without significantly damaging the 

content. 

5 The basic program represents a key part of the invention itself. This program is then 
used as the method by which copyright information is to be associated in an integral 
manner with the content. This is a concept absent from copy protection, encryption 
and content extension schemes. The copyright information itself can be made 
undeniable and unforgeable using cryptographic techniques, so.that through it an 
10 audit traU of ownership my be established for each copy of a given piece of content, 
thus customizing each copy to a particular owner, in a way that can be used to 
identify the owner. 

The value of the stega-dpher is that it provides a way to watermark the content in a 
15 way that changes it slightly, but does not impact human perception significantly. 
And, fiirthennore, that it is made difficuh to defeat since one must know exactly 
where the information resides to extract it for analysis and use in forgery attempts, 
or to remove it without overly degrading the signal. And, to try to forge copyright 
information one must first be able to analyze the encrypted copyright infi>rmation. 
20 and in order to do that, one must be able to find it. which requires masks. 

n. Example Embodiment of General Processing 

Digital audio data is represented by a series of samples in 1 dimension, 

25 

{S,.S2,S3...S„} 



This series is also referred to as a sample stream The sample stream approximates 
an analog wavefonn of sound ampUtude over time. Each sample represents an 
estimate of the wave amplitude at the instant of time the sample is recorded. For 
monaural audio, there is one such sample stream. Stereo audio is comprised of two 
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sample streams, one representing the right channel, and the other lepresradng the 
left. Each stream is used to drive a corresponding speako* to reproduce the stereo 
sound. 

5 What is referred to as CD quality audio is characterized by 16 bit (2 byte) stereo 
samples, recorded at 44. 1 Khz, or 44, 100 samples per second in each channel. The 
dynamic range of sound reproduction is directly proportional to the number of bits 
per sample. Some lower quality recordings are done at 8 bits. A CD audio 
recording can be stored using any scheme for containing the 2 sample streams in 
10 their entirety. When these streams are played back at the same frequency they were 
recorded at, the soimd recorded is reproduced to a high degree of accuracy. 

The sample stream is processed in order from first sample to last. For the purpose 
of the invention disclosed, the stream is separated into sample windows, each of 
15 which has a fixed number of consecutive samples from the stream, and where 

wndows do not overlap in the sample stream. Windows may be contiguous in the 
sample stream. In this discussion assume each A^ndow contains 128 samples, and 
that windows are contiguous. So, the windows ^thin the stream look like 

^® {[Sl> S2, S3...Si28]> [Si29iS|30,Si3|...S236],...[Sn.i28...SJ } 

^ere [...] denotes each window and any odd samples at the end of the stream 
vAiich do not completely fill a window can be ignored, and simply passed through 
the system unmodified. 

25 These windows will be used as input for the discrete Fast Fourier Transform (and 
its inverse) operation. 

Briefly, Fourier Transform methods are based on the principle that a complex 
waveform, expressed as amplitude over time and represented by a sample stream, is 
30 really the sum of a number of simple waveforms, each of which osdUate at different 
fi^uendes. 



PCr/DS9Ma257 

By complex, it is meant that the value of the next sample is not easOy predicted 
from the values of the bst N samples or the time of the sample. By simple it is 
meant that the value of the sample is easily predictable from the values of the hist N 
sanq)les and/or the time of the sam|rie. 

5 

The sum of multiple simple waves is equivalent to the complex wave. The discrete 
FFT and its Inverse simply translate a limited amount of data from one side of this 
equivalence to the other, between the complex waveform and the sum of simple 
waves. The discrete FFT can be used to translate a series of samples representii^ 

10 ampUtude over time (the complex wave, representing a digital audio reconling) into 
the same number of samples representing total spectral energy in a given range of 
frequencies (the simple wave components) at a particuhu- mstant of time. This 
instant is the time in the middle of the original amplitude/time samples. The mverse 
discrete FFT translates the data in the other direction, producing the complex 

1 5 wav^rm, from its simpler parts. 

Each 128 sample window wiU be used as an input to the discrete FFT. resulting in 
128 bins representuig each of 128 frequency bands, ranging from OH2 to 22Kh2 
(the Nyquist frequency, or V4 the samplmg rate). 

20 

Infoimation can be encoded into the audio signal in the fi«quency domain or in the 
time domam. In the latter case, no FFT or inverse FFT is necessary. However, 
encoding m the frequency domain is recommended, since its effects are scattered 
over the resultant time domain samples, and not easily predicted. In addition, 

25 frequency domain encoding makes it more likely that randomization wiU result in 
noticeable artifects in the resultant signal, and therefore makes the stega-dpher 
more defensible against such attacks. It is in the frequency domain that additional 
information wiU be encoded into the audio signal for the purpose of this discussion. 
Each frequency band in a gwen time slice can potentially be used to store a small 

30 portion of some additional infr>rmation to be added to tiie signal. Since tiiese are 
discrete estimates, tiiere is some room for error which wUI not significantiy effect 
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the perceived qualhy of the signal, reproduced after modification, by the inveise 
FFT operation. In effect, intentional changes, which cannot be distinguished from 
random variations are introduced in the frequency domain, for the pmpose of 
storing additional information in die sample stream. These changes are mimmized so 
as not to adversely affect tiie perceived quality of tiie reproduced audio signal, after 
it has been encoded widi additional information in tiie manner described below. In 
addition, tiie location of each of tiiese changes is made virtually impossible to 
predict, an innovation which distinguishes tiiis scheme from simple steganographic 
techniques. 

Note tiiat tiiis process differs from tiie Nagata, et al. patents, 4,979,210 and 
5,073,925, vdiidi encode information by modulatmg an audio signal in 
ampUtude^time domain. It also differs in tiiat tiie modulations mtroduced m tiie 
Nagata process (which are at veiy low amplitude and fi^quency relative to tiie 
canier wave as to remain inaudible) carry only copy/ don't copy information, which 
is easUy found and circumvented by one skilled in the art. Also, there is no 
limitation in tiie stega-dpher process as to what type of infoimation can be encoded 
mto tiie signal, and tiiere is more information storage capacity, since tiie encodmg 
process is not bound by any particular frequency of modulation but ratiier by tiie 
number of samples available. The granularity of encoding m the stega-dpher is 
detennined by tiie sample window size, witii potentially 1 bU of space per sample or 
128 bits per window (a secure implementation will halve tiiis to 64 bits). In Nagata, 
et aL tiie granuhrity of encoding is foced by tiie amplitude and frequency 
modulation limits required to maintain inaudibility. These lunits are rdatively low, 
and tiierefore make it impractical to encode more tiian simple copy/ don't copy 
mformation using the Nagata process. 
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BL Example Embodiment of Encoding and Decoding 

A modification to standard steganographic teduique is applied in the frequency 
domain described above, in order to encode additional information into the audio 
5 signal 

In a sdione adapted from ciyptographic techniques, 2 keys are used in the actual 
encode and decode process. For the purposes of this invention the keys are referred 
to as masks. One mask, the piimaiy, is applied to the frequency a»s of FFT results, 

10 the other mask is appHed to the time axis (this will be called the convolution mask). 
The number of bits comprising the primary mask are equal to the sample window 
dze in samples (or the number of frequency bands computed by the FFT process), 
128 in this discussion. The number of bits in the convolution mask are entirely 
aibitraiy. This implementation will assume a time mask of 1024 bits. Generally the 

15 lai:ger the key, the more difficult it is to guess. 

Prior to encoding, the primaiy and convolution masks described above are 
generated by a cryptographically secure random generation process. It is possible to 
use a block cipher like DES in combination with a sufficiently pseudo-random seed 
20 value to emulate a ciyptographically secure random bit generator. These keys will 
be saved along with information matching them to the sample stream in question in 
a database for use in decoding, should that step become necessaiy. 

Prior to encoding, some additional information to be encoded into the signal is 
tS prepared and made available to the encoder, in a bit addressable manner (so that it 
may be read one bit at a time). If the size of the sample stream is known and the 
efficiency characteristics of the stega-dpher implemertation are taken into account, 
a known limit may be imposed on the amount of this additional infoimation. 

0 The encoder captures one sample window at a time from the sample stream, in 
sequential, contiguous order. The encoder tracks the sequential number of each 
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window it acquires. The first vidndow is 0. When the number of windows processed 
readies the number of bits in the window mask, minus one, the next value of the 
window counts will be reset to 0. 

This counter is the convolution index or phase. In the current implementation it is 
used as a ample index into the convolution bitmadc. In anticipated developments it 
wiU be used to perform convolution operations on the convolution mask to 
determine which bit to use. For instance the mask might by rotated by a number 
corresponding to the phase, in bits to the left and XORed withlhe primary mask to 
produce a new mask, which is then indexed by the phase. There are many 
possibilities for convolution. 

The encoder computes the discrete FFT of the sample window. 

Starting with the lowest frequency band, the encoder proceeds through each band 
to the highest, visiting each of the 128 frequency bands in order. At each band 
value, the encoder takes the bit of the primary mask corresponding to the frequency 
band in question, the bit of the convolution mask corresponding to the window in 
question, and passes these values into a boolean function. This fimction is designed 
so that it has a near perfectly random output distribution. It will return true for 
approximately 50% of its input permutations, and fidse for the other 50%. The 
value returned for a ^ven set of inputs is fixed, however, so that it will always 
return the same value ^ven the same set of inputs. 

If the function returns true, the current frequency band in the current window is 
used in the encoding process, and represents a valid piece of the additional 
information encoded in the signal. If the function returns &lse, this cell, as the 
frequency band m a given window is called, is ignored in the process. In this manner 
it is made extrrafiely difficult to extract the encoded information from the agnal 
wthout the use of the exact masks used in the encoding process. This is one place 
in which the stega-cipher process departs from traditional steganographic 
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implementatioiis, wMch offer a trivial decode opportunity if one knows the 
information is present. While this increases the information storage capacity of the 
carrier signal, it makes decoding trivial, and further degrades the signal. Note that it 
is possible and desirable to modify the boolean cell flag function so that it returns 
5 true < 50% of the time. In general, the fewer cells actually used in the encode, the 
more diflScult they will be to find and the less degradation of content will be caused, 
provided the function is designed correctly. There is an obvious tradeoff in storage 
capadty for this increased security and quality. 

10 The encoder proceeds in this manner until a complete copy of the additional 

infoimation has been encoded in tiie carrier signal. It wiU be desirable to have the 
encoder encode multiple copies of tiie additional information continuously over tiie 
duration of tiie carrier signal, so tiiat a complete instance of this information may be 
recovered fiom a small«- segment of a larger signal which has been split into 

15 discontinuous pieces or otiierwise edited. It is therefore desirable to mmimize tiie 
size of the information to be encoded using both compact design and pre-encoding 
compression, thus maximizing redundant encoding, and recoverability fi^om smaller 
segments. In a practical implementation of tius system it is likely tiie information 
wUI be first compressed by a known metiiod, and then enaypted using public-key 
20 techniques, before being oicoded into tiie carrier signal. 

The encoder wiD also prepare tiie package of additional information so tiiat it 
contiiins an easUy recognizable start of message delimeter, which can be unique to 
each encoding and stored along with the keys, to serve as a synchronization signal 

25 to a decoder. The detection of this delimeter in a decoding window signifies tiiat the 
decoder can be reasonably sure it is aligned to tiie sample stream correctiy and can 
proceed in a metiiodic window by window manner. These delimeters wiU require a 
number of bits which minunizes tiie probability tiiat tiiis bit sequence is not 
reproduced in a random occurrence, causing an accidental misalignment of tiie 

30 decoder. A minimum of 256 bits is recommended. In tiie cuirent implementatic 
1024 bits representing a start of message delimeter are used. If each sample is 
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nmdom, then each bit has a 50% probably of matching the ddimeter and the 
conditional probability of a random match would be in"". In practice, the samples 
are probably somewhat less than random, increasing the probability of a match 
somewAiat 

5 

The decode process uses the same masks in the same manner, only m this case the 
information is extracted one bit at a time from the carrier signal. 

The decoder is assumed to have access to the proper masks used to encode the 
10 information originally. These masks might be present in a database, wluch can be 
mdexed by a value, or vahies computed from the original content, in a manner 
ulsenative to the modifications to the content caused by the st^ga-dpho- process. 
So, ^en an aibitraiy piece of content, a decoder might first process the contoit to 
generate certain key values, and then retrieve the decode masks assodated with the 
15 matching key values from the database. In the case where multiple matches occur, 
or none are found, it is conceivable that all mask sets in the database could be tried 
sequentially until a vaUd decode is achieved, or not, indicating no infonnation is 
present. 



20 In the application of this process, it is antidpated that encodmg operations may be 
done on a ^ven piece of contwit up to 3 times, each adding new informadon and 
udng new mask% over a sub-segment of the content, and that decode operations 
will be done uifrequently. It is antidpated that should it become necessary to do a 
search of a laige number of masks to find a valid decode, that this process can be 

25 optimized using a guessing technique based on close key matching, and that it is not 
a time critical application, so it Avill be feasible to test large numbers of potential 
masks for validity on a given piece of content, even if such a process takes days or 
weeks on powtffid computers to do a comprehensive search of known mask sets. 

30 The decode process is slightiy different m the following respect. Whereas the 
encoding process can start at any arbitraiy point in the sample stream, the dwode 
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process does not know vihere the mcode process began (the exact offset in samples 
to the start of the first window). Even though the encode process, by convention, 
starts with sample 0, there is no guarantee that the sample stream has not been 
edited ance encodmg, leaving a partial window at the start of the sample stream, 
and thus requiring the decoder to find the first complete window to start the 
decode. Therefore, the decode process wU start at the first sample, and shift the 
sample window along by 1 sample, keeping the window index at 0, until it can find 
a valid decode delimeter encoded m the window. At this point, the decoder knows 
it has synchronized to the encoder, and can then proceed to process contiguous 
windows in a more expedient manner. 

Example Calculations based on the described implementation for adding copyright 
certificate information to CD quality digital audio: 

In a stream of samples, every 128 samples will contain, on average 64 bits of 
certificate related information. Digital audio is composed of 16 bit samples, at 44. 1 
Khz, or 44,100 samples per second. Stereo audio provides 2 streams of information 
at this rate, left and righjt, or 88,200 samples per second. That yields approxunately 
689 contiguous sample windows (of 128 samples) per second in which to encode 
information. Assume a song is 4 minutes long, or 240 seconds. This yields 240 ♦ 
689 = 165,360 windows, which on average (50% utilization) contain 64 bits (8 
bytes) each of certificate information. This in turns gives approximately 1291Kb of 
information storage space per 4 minute stereo song (1.2 MB). There is ample room 
for redimdant encoding of information continuously over the length of the content. 
Encoding 8 bytes for every 256 bytes represents 3.1% of the signal information. 
Assuming that a copyright certificate requires at most approximately 2048 bytes 
(2K), we can encode the same certificate in 645 distinct locations within the 
recording, or approximately every 37/lOOths of a second. 

Now to account for delimeters and synchronization information. Assuming a sync 
marker of 1024 bits to avoid random matches, then we could prefix each 2K 
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certificate block with this 1024 bit marker. It takes 256 windows to store 2K, and 
under this proposed scheme, the first 1 6 windows are reserved fi>r the sync marker. 
A decoder could search fijr this marker by progressivdy matchiiig eadi of the first 
16 windows (64 bits at a time) against the conresponding portion of the sync 
5 marker. The decoder could reset the match advancing through the sample stream, 
as soon as one window did not conform to the sync maricer, and proceed in this 
manner until it matches 16 consecutive windows to the marker, at which point it is 
syndironized. 

10 Under this scheme, 240 windows, or 1.92K remain for storing certificate 
infi)nnation, whidi is not unreasonable. 

IV. Possible Problems, Attacks and Subsequent Defenses 

15 A. Randomization 

The attacker simply randomizes the least significant bits of each data point in the 
transform buflfer, obliterating the synchronization signal and the watermark. While 
this attack can remove the watermark, in the context in which stega-cipher is to be 
used, the problem of piracy is kept to a minimum at least equal to that afforded by 

20 traditional mecfia, ance the system will not allow an unwatermarked piece of 

content to be traded for profit and watermarks cannot be forged without the proper 
keys, which are computationally difficult to obtain by biute-force or cryptanalysis. 
In addition, if the encoding is managed in sudi a way as to maximize the level of 
changes to the sample stream to be just at the threshold below human perception, 

25 and the scheme is implemented to anticipate randomization attempts, it is possible 
to force the randomization level to exceed the level that can be perceived and cr«ate 
destructive artifacts in the signal, in much the same manner as a VHS cassette can 
be manufactured at a minunal signal level, so that a single copy results in 
unwatchable static. 
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B. Low Bit-Depth Bitmaps (black & white images) 

These bitmaps would be too sensitive to the steganization process, resulting in 
unacceptable dgnal degradation, and so are not good candidates for the stega- 
cipher process. The problmi may be circumvented by inflating bit-dq)th, although 
5 this is an inefficient use of space and bandwidth. 

C. Non-Integer Transforms 

The FFT is used to generate spectral energy information for a given audio signal. 
This information is not usually in integer format. Computers use methods of 

10 approxuniation in these cases to represent the real numbers (whole numbers plus 
fiactional amounts). Depending on the exact value of the number to be represented 
slight errors, produced by roundmg off the nearest real number that can be 
completely specified by the computer occur. This will produce some randomization 
m the least significant bit or bits. In other words, the same operation on the same 

15 sample window might yield slightly different transform values each time. It is 
possible to circumvent this problem using a modification to the simple LSB 
st^anographic technique described later. Instead of looking at the LSB, the stega- 
dpher can use an energy quantization technique in place of the LSB method. Some 
variant of rounding the spectral energy values up or down, with a granularity 

20 greater than the rounding error should work, v^thout significantly degrading the 
output samples. 

V. A Method and Protocol For Using the Stega-Cipher 

25 The apparatus described in the claims below operates on a window by window basis 
over the sample stream. It has no knowledge of the nature of the specific message 
to be encoded. It m^ely indexes into a bit stream, and encodes as many of those 
bits as possible into a given sample window, using a map determined by the given 
masks. 



30 
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The value of encoding information into a single window in the sample stream usmg 
sudi an apparatus may not be inhmntly apparent until one exammes the maimer in 
iKdiich such information will be used. The protocol discussed in this section details 
how messages which occeed the encoding capacity of a single sample window (128 
5 samples) may be assraibled fix)m smaller pieces encoded in the individual windows 
and used to defend copyrights in an online situation. 

An average of 64 bits can be encoded into each window, which equals only 8 bytes. 
Messages larger than 8 bytes can be encoded by simply dividing the messages up 
10 and encoding small portions into a string of consecutive windows in the sample 
stream. Since the keys determine exactly how many bits will be encoded per 
window, and an element of randonmess is desirable, as opposed to perfect 
predictability, one cannot be certain exactly how many bits are encoded into each 
wmdow. 

15 

The start of each message is marked by a special start of message delimeter, which, 
as discussed above is 1024 bits, or 128 bytes. Therefore, if precisely 8 bytes are 
encoded per window, the first 16 windows of any useable message in the system 
described here are reserved for the start of message delimeter. For the encoder, this 

20 scheme presents little challenge. It ^mply designates the first sample window in the 
stream to be wndow 0, and proceeds to encode the message delimeter, bit-by-bit 
mto each consecutive window. As soon as it has processed the last bit of the SOM 
delimeter it continues by encoding 32 bits representing the size, in bytes of the 
complete message to follow. Once the 32nd and final bit of the size is encoded, the 

25 message itself is encoded into each consecutive window, one bit at a time. Some 
windows may contain more encoded bits then others, as dictated by the masks. As 
the encoder processes each window in the content it increments its window counts. 
It uses this counter to index into the window mask. If the number of windows 
required to encode a complete message is greater than the ^e of this mask, 256 

30 bits m this case, or 256 windows, then it simply resets the counter aftea- window 
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2SS, and so on, until a complete message is mcoded. It can then start over, or start 
on a new message. 

The decoder has a bigger challenge to &ce. The decoder is given a set of masks, 
5 just like encoder. Unlike the encoder, the decoder cannot be sure that the first series 
of 128 samples it receives are the window 0 start of message, encoded by the 
decoder. The sample stream originally produced by an encoder may have been 
edited by dipping its ends randomly or splicing pieces together. In that case, the 
particular copy of the message that was clipped is unrecoverable. The decoder has 
10 the start of message ddimeter used to encode the message that the decoder is 
looking for. In the initial state, the decoder assumes the first window it gets is 
window 0. It then decodes the proper number of bits dictated by the masks it was 
gh^n. It compares these bits to the corresponding bits of the start of message 
delimeter. If they match, the decoder assumes it is still aligned, increments the 
15 window counter and continues. If the bits do not match, the decoder knows it is not 
aligned. In this case, it shifts one more sample onto the end of the sample buffer, 
discarding the first sample, and starts over. The window counter is set to 0. The 
decoder searches one sample at a time for an alignment lock. The decoder proceeds 
in this manner until it has decoded a complete match to the start of message 

20 delimeter or it exhausts the sample stream wthout decoding a message. If the 
decoder can match completely the start of message delimeter bit sequence, it 
switches into aligned mode. The decoder will now advance through the sample 
stream a full window at a time (128 samples). It proceeds until it has the 32 bits 
specify^g the message size. This generally won't occupy more than 1 complete 

25 window. When the decoder has locked onto the start of message delimeter and 
decoded the message size, it can now proceed to decode as many consecutive 
additional windows as necessaiy until it has decoded a complete message. Once it 
has decoded a complete message, the state of the decoder can be reset to un- 
synchronized and the entire process can be repeated starting with the next 128 

30 sample window. In this manner it is not absolutely necessary that encoding windows 
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be contiguous in the sample stream. The decoder is capable of handling random 
intervals between the end of one message and the start of another. 

It is unportant to note that the drcuit for encoding and decoding a sample window 
S does not need to be aware of the nature of the message, or of any structure bqrond 
the start of message delimeter and message siz/d. It only needs to consider a single 
sample window, its own state (t^ether the decoder is misaligned, synchronizing, or 
synchronized) and what bits to racode/decode. 

10 Given that the stega-cipher apparatus allows for the encoding and decoding of 
arbitrary messages in this manner, how can it be used to protect copyri^ts? 

The most important aspect of the st^-cipher in this respect is that feet that it 
makes the message intend with the content, and difficult to remove. So it cannot 
15 be eliminated simply by removing certam information prepended or appended to the 
sample stream itself In fact, removing an arbitrary chunk of samples will not 
generally defeat the stega-cipher either. 

Given tiiat some information can be thus integrated with the content itself the 
20 question is then how best to take advantage of this arrangement in order to protect 
copyrights. 

The following protocol details how the stega-cipher will be exploited to pix>tect 
copyrights in the digital domain. 

25 

In a transaction involving the transfer of digitized content, there are at least 3 
functions involved: 

The Authority is a trusted arbitrator between the two otiier functions listed below, 
30 representing parties who actually engage in the transfer of the content. The 

Authority maintains a database contairung information on the particular piece of 
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content itself and who the two parties engaged in transferring the content are. The 
Authority can perfbnn st^-cipher encoding and decoding on contoit 



The Publisher, or online distributor is the entity which is sending the copyrighted 
5 content to another party. The Publisher can perform st^-dpher encoding and 
decoding on content. 

The Consumer is the person or entity receiving the copyrighted content, generally in 
occhange for some consideration such as money. The conaimer cannot genially 
10 perform st^-cipher mcoding or decoding on contmt. 

Each of these parties can participate in a message exdiange protocol using wdl 
known public-key ciyptograpluc techniques. For instance, a ^em licensing RSA 
public key algorithms might be used for signed and enciypted message exchange. 
15 This means that each party maintains a public key / private key pair, and that the 
public keys of eadi party are freely available to any other party. Generally, the 
Authority communicates via electronic links directly only to the Publisher and the 
Consume- communicates directly only with the publisher. 

20 Below is an example of how the protocol operates from the time a piece of content 
enters an electronic distribution system to the time it is delivered to a Consumer. 

A copyright holder (an independent artist, music publisher, movie studio, etc.) 
wishes to retaU a particular title online. For instance. Sire Records Company might 

25 wish to distribute the latest single from Seal, one of their musical artists, online. Sire 
deUvers a master copy of this single, "Prayer for the Dying", to the Authority, 
Ethical Inc. Ethical converts the title into a format suitable for dectronic 
distribution. This may involve digitizing an analog recording. The title has now 
become content in the context of this onUne distribution system. The title is not yet 

30 available to anyone except Ethical Inc., and has not yet been encoded with the 
st^-dpher watermark. Ethical generates a Title Identification and Authentication 
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(TIA) certificate. The cotificate could be in any fonnat. La this example it is a short 
text file, readable with a small word-processing program, whidi contains 
information identifying 

5 the title 

the artist 

the copyright holder 

the body to wAich royalties should be paid 
general terms for publishers' distribution 
10 any other mformation helpful in identifying this content 

Ethical then agns the TIA with its own private key, and «icrypts the TIA certificate 
plus its signature with its own public key. Thus, the Ethical can decrypt the TIA 
COTtificate at a later time and know that it generated the message and that the 
15 contents of the message have not been changed since generation. 

Sire Records, which ultimately controls distribution of the content, communicates 
to the Ethical a specific online Publisher that is to have the right of distribution of 
this content. For instance, Joe's Online Emporium. The Authority, Ethical Inc. can 
20 transmit a short agreement, the Distribution Agreement to the Publisher, Joe's 
Online Emporium v/tadi lists 

the content title 
the publisher's identification 
25 the terms of distribution 

any consideration paid for the right to distribute the content 
a brief statement of agreement with all terms listed above 



30 



The Publisher receives this agreement, and signs it using its private key. Thus, any 
party with access to the Joe's Online Emporium's public key could verify that the 
Joe's signed the agreement, and that the a^eement has not been changed since 
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Joe's signed it The Publisher transmits the signed Distribution Agreement to the 
Authority, Ethical Inc. 

Ethical Inc. now combines the signed HA certificate and the Distribution 
5 Agreement mto a angle message, and agns the entire message using its private key. 
Ethical has now created a Publisher Identification message to go into its own stega- 
cipher channel in the content. Ethical Inc. now generates new stega-cipher masks 
and encodes this message mto a copy of the content using a stega-cipher encoder. 
The Authority saves the masks as a Receipt in a database, along with infoimation 
10 on the details of the transfer, including the title, artist and publisher. 

Ethical ften transfers this watermariced copy to the Joe's Online Emporium, the 
Publisho-. Well known enayption methods could be used to protect the transfer 
betweoi the Authority and the Publisher. The Authority may now destroy its copy, 
15 which the Publisher has received. The Publisher, Joe's Online Emporium now 

assumes responsfl>ility for any copies made to its version of the content, which is a 
Publisher Master copy. 

Finally, the Consume-, John Q. Public wishes to purchase a copy of the content 
20 fi-om Joe's Online Emporium. Joe's Emporium sends the John Q. Public a short 
agreement via an electronic conmiunication link, similar to Publisher's Distribution 
Agreem^tt, only this is a Purchase Agreement, vMch lists 



the content title 
25 consumer identification 

the terms of distribution 
the consideration pas for the coment 
a brief statement of agreement with the terms above 



30 John Q. Public signs this agreement with his private key and returns it to the Joe's 
Online Emporium. The Publisher, Joe's prepares to encode its own stega-cipher 
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watennark onto a copy of the content by generating a set of masks for the 
algorithm. Joe's Online Emporium then stores these masks (a recdpt) in its own 
database^ indexed by title and consumer. Joe's Online Emporium dgns the 
agreement recdved from John Q. Public with the Emporium's own private key, and 
5 forwards it to the Authority, Ethical Inc., along with a copy of the masks. It is 
important to note that this communication should be done over a secured channel. 
The Authority verifies the Publisher and Consumer information and adds its own 
signature to the end of the message, approving the transaction, creating a Contract 
of Sale. The Authority adds the Publisher's receipt (mask set) to its database, 

10 indexed by the title, the publisher, and the consumer identification. The Authority 
agns the Contract of Sale by encrypting it with their private key. So anyone with 
the Authority's public k^ (any Publisher) could decrypt the Contract of Sale and 
verify it, once it was extracted from the content. The Publisher then transmits the 
signed Contract of Sale back to the Publisher, who uses a stega-cipher device to 

15 imprint this Contract as its own watermark over the content. The Publisher then 
transmits the newly watermarked copy to the Consumer, who is accepting 
responsibility for it. The Publisher destroys their version of the consumer's copy. 

If this procedure is followed for all content distribution within such an online system 
20 then it should be pos^ble for the Authority to identify the owner of a piece of 
content which appears to be unauthorized. The Authority could ^ply try its 
database of st^-dpher keys to decode the watermark in the content in question. 
For instance, if a copy of Seal's latest single originally distributed with stega-cipher 
watermarics showed up on an Internet ftp site the Authority should be able to 
25 extract a TIA Certificate and Distribution Agreement or a Contract of Sale 
identifying the responsible party. If a Publisher sold this particular copy to a 
Consumer, that particular publisher should be able to ^act a Contract of Sale, 
which places responsibility with the Consumer. This is not a time critical 
iq)plication, so even if it takes days or weeks, it is still worthwhile. 



30 
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Id a modification to the protocol discussed above, each Publisher mi^ act as its 
own Authority. Howevo-, in the context of online services, this could open avmies 
of fiaud conunitted by the collusion of certain Publishers and Consumers. Uang an 
Authority, or one of several available Authorities to keep records of Publisher* 
5 Ck>nsumer transactions and verify their detdls decreases the likelihood of such 
evoits. 



It should also be obvious that a similar watermarking system could be used by an 
individual entity to watermark its own contait for its own purposes, wether online 

10 or in phyacal media. For instance, a CD manufacturer could incorporate unique 
st^-cipher watermarks into spedfic batches of its compact discs to idoitify the 
source of a pirate ring, or to identify unauthorized digital copies made from its 
discs. This is possible because the stega-cipher encoding works with the enstmg 
formats of digital samples and does not add any new structures to the sample data 

15 that cannot be handled on electronic or mechanical systems which predate the 
stega-cipher. 

VL Increasing Confidence in the Stega-Cipher 

20 The addition of a special pre-encoding process can make st^-cipher cotificates 
even more secure and undeniable. Hash values may be incorporated whidi ma^ 
cxacdy the content containing the wat«mark to the message in the watermark 
itself This allows us a verification that the watermark decoded was encoded by 
iKiiomever signed it into this precise location in this specific content. 

25 

Suppose one wants to use a 256 bit (32 byte) hash value which is calculated with a 
secure one-way hash fimction over each sample in each sample window that will 
contain the message. The hash starts with a seed value, and each sample that would 
be processed by the encoder v/hm encoding the message is mcorporated into the 
30 hash as it is processed. The result is a 256 bit number one can be highly confident is 
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unique^ or sufiSciently rare to make intmtionally duplicating it with another series of 
sanq)les difBcutt 



It is important that the hash function be insensitive to any changes in the samples 
5 induced by the st^-cipher itself. For instance, one might ignore the least 

agnificant bit of each sample when computing the hash function, if the stega-cipher 
was implemented using a least significant bit encode mode. 

Based on the size of the non-hash message, one knows the hash-inclusive message 
10 requires 32 more bytes of space. One can now calculate the size of a ^gned 

enoypted copy of this message by signing and encrypting exactly as many random 
bytes as are in the message, and measuring the size of the output in bytes. One now 
knows the ^ze of the message to be encoded. One can pre-process the sample 
stream as follows. 

IS 

Proceed through the stega-cipher encode loop as described in the claims. Instead of 
encoding, however, calculate hash values for each window series which will contam 
the message, as each sample is processed. At the end of each instance of '^encoding** 
take the resultant hash value and use it to create a unique copy of the message 
20 vMch includes the hash value particular to the series of sample windows that will be 
used to encode the message. Sign and enciypt this copy of the message, and save it 
for mcoding in the same place in the sample stream. 

A memory efiScient version of tMs scheme could keep on hand the un-hashed 
25 message, and as it creates each new copy, back up in the sample stream to the first 
window in the series and actually encode each message, disposing of it aftawards. 

The important result is evident on decoding. The decoding party can calculate the 
same hash used to encode the message for themselves, but on the encoded samples. 
30 If the value calculated by the decoding party does not match the value contained in 
the dgned message, the decoder is alerted to the fact that this watermark was 
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transplanted from somewhat dse. This is possible only with a hash function vAadk 
igaoT^ the changes made by the st^-dpher aftw the hash in the watermark was 
generated. 

5 This scheme makes it impossible to transplant watennarks, even with the keys to 
the stega-cipher. 
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Appendix - Psuedo-code 

const int WINDOW^RESET = 256; 
const int WINDOW^SIZE « 128; 
const int MARKERBITS » 1024; 
const int CHUNK.BITS " 2048 * 8; 

intivindow_oi!t^ 
intnisg_lHt_oflsc^ 
int firequency_ofrset^ 
Boolean iiseCcdl; 

/• 8 bits per 1^ J byte per char */ 
unagncd char frcqucncy_maskIWINDOW_SIZE/8J; 
unsigned char window_niask[WINDOW_RESET/8]; 
unsigned char msg_start_marker[MARKER_BrrS/8]; 
unsigned char msg_end_markcr[MARKER_BITS/81; 
Intl6 ampfitude_sample_buffiETfWINDOW_SIZE]; 
float power.frequency_bufrer[WINDOW_SIZE); 
unagned char message.bufTerfCHUNK.BrTS/S]; 

void doFFT(IiU16 •amp_saniplc_buflcr, float *powcr^fre<Lbufrcrjnt sire); 

void dQlnverseFFT(Int]6 •amp^sample^buficr, float •power Jreq:^bufler,int sizcX 

voidtnitializea 

Bit getBit(unsigned char *bufrer,int bitOflsetX 

Boolean map(Bit window_bit. Bit band^Wt, int window, int frequency); 

Boolean getSamples(Intl6 ♦amplitude_sample__buflerjnt samples^ 

voidenoodcO 

vofdinitializoO 
{ 

/♦ message to be encoded is generated ♦/ 

/• message is preflxed with 1 024 bit msg^start^markcr ♦/ 

/* message is sufGxed with 1024 bit msg^end _marker */ 

/• remaining space at end of message buffer padded with random bits •/ 

window_oi!8et " 

nisg_bit_o£&ct«0; 

frequency_ofisct » 0; 

frequcncy_mask loaded 

window^mask loaded 

zeroAmpSampleBufreiO; 
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Botdean gelSamples(IntI6 *bufra>4nt samples) 
{ 

/• get samples number of samples and shift them contiguously into the sample 

buffer from light to teft*/ 
Utsamples <8aiiq]|es available) 
. return fidse; 

else 

letunitniQ 

} 

void d6Frr(Intl6 ♦sample.buffcr, float •spcctrum.buffer, int size) 
{ 

calculate FFT on sample^buff^, for size samples 
store result in spectrum buffer 

> 

void doInven»IOT(Intl6 •sample.buffer^teat ♦spectrum^^^ 
{ . 

calculate inverse FFT on spectnim_buffer 

store result in sampe^buffcr 

} 

Bit getBit(unsigned char *buffer^n bitOfiset) 
{ 

returns value of q3ecified bit in specified buffer 

either 0 or 1 , could use Boolean (truc/felse) values for bit set of bit off 

> 

Boolean map(Bit window_bit3it band^bit jnt window, int frequency 
{ 

this is the function that makes the infoimation difficult to find •/ 
/• the inputs window^bit and band^bit depend only on the mask values 

used for encoding the information, they are 1) nuidom, 2) seciet •/ 
/* window and freqtKocy values are used add time and frequency band dependent 

comidexity to this function V 
/• this function is equivalent to a Boolean truth table with window • frequency » 4 
possible input oombiiiations and 2 possible output V 
/* for any input combination, the output is dther true or false •/ 
/'window ranges ffomO to WINDOW^RESET-l •/ 
/♦ firqucmy ranges from 0 to WINDOW^SEE . I •/ 
return calculated truth value 
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void enoodeBtt(ibat ^spectnim.bufier^nt ifeq_ofrset3it theBit) 
{ 

A* modifies the value of the cell inspectrum.lnifler, indexed by ficq^offot 

m a manner that distinguishes each of the 2 |x»si^le val 
lorO 

•/ 

/♦ suggested method of setting the Least Significant bit of the cell theBit •/ 
/• alternative method of rounding the value of the cell upward or downwaid to 
certain fiactional vahies pioposed 

ix. <^ 3 fiactiomd lenuurider signifies 0, > .5 fraction remaind^ 
sfgmfiesl 

•/ 

} 

votdencodeO 
{ 

initialize^. 



do{ 

if{getSamples(ampHtude_sample_bufrer) false) 
return 

dQFrr(amplitude_sample buffer jower frequency bufier.WPmnw jglZP )' 

for (frequency^offset « 0; frcqucncy^ofrsct < WINDOW^SIZE; 
frequen^joffset-f-f-X 

useCdl *= map(geffiit(window_mask,window_ofrset}, 

gefBit(frequency_nnask/requem^_ofrsetX 
window^oflset, fjequenQLoifset); 

if{useCeII«»tnie){ 

encodcBit(powcr_frequency_bufrcr/rcqucncy_o£rsct, 

getBit(message_bufrer,m^btt_ofrset}); 
message_bit_ofl&et ++; 
ifl[msg_bit^ofrsct MESSAOEBrrS){ 

initialize(X 

break; /♦ exit frequency loop ♦/ 

) 

) 

) 
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d(^vefseiFFT(aiiii£tiide_8ami^^ 
TONDOW.SIZE); 

oiit|mtSamples(ampliti^^ 

window_o£rset++; 

if(windaw_ofiOset — WINDOWJffiSET){ 
wimlafWjof&ct'^O; 

> 



}whilc(tnieX 

} 

The enoodeO pioccduic pitxxsses an input sample stream using the speciHed frequenqr and window madcs as 
well as a pie-foraiatted message to encode. 



encodeO processes the sample stream in windows of WINDOW^SIZE samples, contiguously distributed in the 
sample stream, so it advances WINDOW_S]ZE samples at a lime. 

For each samite window, cncodeQ first compute the FFT of the window, yielding its Power Spectrum Estimation. 
For each of these window PSEs, encode() then uses the map() function to detemii ne where in each PSE to encode 
Ae bits of tiie mess^, which it reads fiom the message buffer, on ebil at a time. Each lime mapQ returns true, 
eneodeO consumes another sampte from the message. 



After each window is encoded, encodcQ computes the inverse FFT on the PSE lo generate a modified sample 
window, which is then output as the modified signal. It is important the sample windows NOT overlap tn the 
sample stream, since this would potentially damage the prececding encoding uindows in the stream. 



Once the message is cnitirely encoded, mcluding its special end of message marker bit stream, encodeQ resets it 
internal variables to begin encoding the message once more in the next window. encodcQ proceeds in this manner 
until the input sample stream is exhausted. 



enum { 

Synchronizing, 
, Locked 
decode states*/ 
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unagned char message_aKl_buffei{MARKER_BITSl; 

Bit decod eBit(fbat ^spectnim^bufibr^t freq^ofTset) 
{ 

/• reads the value of the cell in spectrum_biiffer, indexed by fiecLof&et 

in a manner th{it distinguishes each of the 2 possible values c^an 
encoded bit, I or 0 

•/ 

/* suggested method of testing the Least Significant bit of the cell */ 
/* alternative method of checking the value of the cell versus certain (radional 
remaindeni pioposcd. 

Le. <- .5 &acliona] remainder signifies 0, > .5 fraction remainder 
signifies 1 

♦/ 

return either 1 or 0 as appropriate 

) 

Boolean decodeO 
{ 

/* Diitialization */ 
state Synchronizing 
wtndow_of!set " 
set frequency mask 
set window mask 
dear sample buffer 
intnextSamples^ 1; 
int msg_start_of!set » 0; 
dear message.end^bufTer 
BitaBin 

Boolean bitsEqual; 
do{ 

instate SynchronizingX 
ncxtSamples= 1; 
window^ofTset^O; 

} 

else 

nexiSamples » WINDOW_SIZ£; 

iii[getSamples(amplitude_sample_buirer) = false) 
return false; 
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doFFT(amptitiJde_8ainpte_bufrer4xiwer_fi^equen^^ 
WINDOW_SlZE); /•2*/ 

for (frequency j)f&et « 0; ffcquencyjofiset < WINDOW^SIZE; 
frequencyjof&et-H-X 

useCelt « map(gctBit(wimk>w^inask,wmdow_c^!8etX 
gelBil(frequency_niask/requency_ofisetX 
windowoffsct, frequency_ofl8etX 

if(useCeU""tnie){ 

aBit « decod^it(power.frequem7 bt^^ ' 

frequency^ofEsetX 
se(Bit(message_bufrer^es$age_bit_ofrseUBit); 
mcssagc_bit_offset ++; 

} 

dse 

continue; 
instate Synchionizing){ 
bitsEqual*" 

compareBits(messagc_start_markcr;mcssagc_bufrcr, 

if(;!bitsEqiial){ 

messa^_bit_oir8^ *» 0; 

mtsaltgned ■» true; 

break; exit frequency loop */ 

} 

dsc if (message^bit^ofiaee — MARKER^BfrS) 
state Locked; 

) 

else{ 

/♦ locked onto encoded stream ♦/ 

shifl aBit into right side of message^end^buffer 

UtsEqual « compareBits(message_end_bufrer, 

nisg_end,inarkcr,MARKER_BITSX 
tf)[bitsEquaI) 

return trur, 

> 

} 



} while (trueX 

} 
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The decodeO procedure scans an input sample stream using specified window and fiequency masks, untO it ettfaer 
deeodes a valid message Mock, storing it in a message bufTCT^ 

The deoodeO pioceduie starts in state Synchronizing, in which it does not know where in the sample stream the 

encoding wiiidows are aligned. The procedure advances the 

sample at a time. perfotmiiQ the FFT calculation on each window, and attm 

fiom the window. As it extracts each bit using the mapQ function, the dccodeQ piocdurc compares these Ixts 
against the start of incssagc marker. As soon as a misnwtch is detected, the decode^ 
properly aligned to an encoding window, and immediately ceases decoding bits from the current window and 
moves to the next window, d&et by 1 sample. The dccodeQ procedure continues in this manner until it matches 
successfully the complete bitstream of a start of message marker. At this point the dccodeQ procedure assumes it is 
aligned to an encoded message and can then decode bits to the message bulTer quickly, advancing the sample 
window fiilly at each iterations. It is now in Locked mode. For each bit it stores in the message buffer when in 
Locked mode, the dccodeQ procedure also shifls the same bit value into the least agnificant bit of the 
message.end.bufifer. After each bit is decoded in Locked mode, the decodcQ procedure checks compares the 
message_end_bufier with the msg_end_marker in a bit by bit manner. When a complete match is found, decodeQ 
is finished and returns true. If the sample stream is exhausted before this occurs, decodeQ returns false. If decodeQ 
returns tnie, a valid message is stored in the message buffer, induding tte 
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Oaims 

1 . A st^anographic method comprising the steps of : 

using random keys in combination with steganogiaphy to encode additional 
infonnation into digitized samples such that a signal generated fiom the modified 
sample stream is not significantiy degraded and such that the additional infonnation 
cannot be extracted without the keys and such that the signal generated from the 
modified sample stream will be degraded by attempts to erase, scramble, or 
otherwise obliterate the encoded additional information. 

2- An apparatus for encoding or decoding a message, rqiresented as 

series of data bits into or out of a series of digitized samples, comprising: 

a) a sample buffer for holding and accessing and tiansfonniog 
digitized samples; 

b) a digital signal processor capable of performing &st fourier 

transforms; 

c) a memory to contain information representing 

1) primary mask, 

2) convolutional mask, 

3) start to message delimiter, 

4) a mask calculation buffer, 

5) a message buffer, 

6) an integer representing a message bit index, 

7) a position integer M representing message size, 

8) an integer representing an index into said primary 
mask, 

9) an integer representing an index into said convolution 
mask, 

10) an integer representing the state of a decode process, 

11) a table representing a map function; 

12) a flag indicating a complete message has been 
decoded or encoded. 
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13) a positive integer S rq)resenting a number of samples 
to read into said sample buffer, and 

14) a flag indicating the azeofa message ^ch has been 
decoded; 

d) an input to acquire digital samples; 

e) an output to output modified digital samples; 

f) an input for inputting the values of (cl) - (c5) and (cl 1) and 

(cl3); 

g) an output to output the message stored in (c5) as the result 
of a decode process and the value of (cIO) to an attached digital circuit; 

h) at least one data bus to transfer information from 
(d)to(a), 

(a) to(b), 

(b) to(a), 
(a)to(e), 

(f) to (c), and 

(c) to (e); and 

i) a clock which generates a clock signal to drive (b) and 
control the operation of the apparatus. 

3. A method of encoding information into a sample stream of data, said 
method comprising the steps of 

A) generating a mask set to be used for encoding, said set 

including: 

a random or pseudo-random primary mask, 
a random or pseudo-random convolution mask, 
a random or pseudo-random start of message 

delimiter, wherein said mask set can be concatenated and manipulated as a dngle bit 

stream; 

B) obtaining a message to be encoded; 
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C) generating £ message bit Steam to be mcoded such that the 
stream includes 

1) a start of message ddimiter, and 

2) an integer representing the number of message 
bytes to follow the message; 

D) loading the message bit stream, a map table, the primary 
mask, the convolution mask, and the start of message delimiter mto a memoiy, 

E) resetting a primary mask index, a convolution mask and 
message bit ind^ and setting the message size integer equal to the total numb^ of 
bits in the message bit stream; 

F) clearing a message encoded flag; 

G) reading a window of samples from a sample input device 
and storing them sequentially in a sample buffer; 

H) resetting the primary mask index and looping through the 
sample buffer from a first sample to a last sample incrementing the primary mask 
uxdex each time a sample is visited, such that for each sample position, a value of 
the mappmg function is computed, which is either true or fidse, by usmg a bit of the 
primary mask representing a current sample and a bit of the convolution mask 
mdicated by the convolution index to calculate an ofi&et in the map table; 

J) obtaining the bit value stored in the map table and encoding 
the bit of the message indicated by the message bit index into the current sample if 
the bit value obtained from the map table is a certain value and incrementing the 
message bit index, determining whether the message bit index equals the number of 
message bits, and if it does re-performing step A), setting the message encoded flag, 
and exiting the loop; 

J) outputting the modified samples in the sample buffer, and if 
the message encoded flag is set jumping back to said step E); 

K) incrementing the convolution index, wherein if the 
convohition index equals the length of the convolution mask in bits then set the 
convolution index to 0; and 
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L) jumping back to step G). 

4. A method of encoding information into a sample stream of data, conq)rising 
the steps of: 

S A) generating a mask set to be used for encoding, including: 

a random or pseudo-random primary mask, 
a random or pseudo-random convolution mask, and 
a random or pseudo-random start of message 
delimiter, wherein said mask set can be concatenated and manipulated as a single bit 
10 stream; 

B) inputting a message to be encoded; 

C) generating a message bit stream to be encoded mcluding 
a start of message delimiter, and 

an integer representing of number of message bytes to 
15 follow the message; 

D) loading the message bit stream, a map table, and the mask set 

into a memory; 

E) resetting a primary mask index, a convolution mask and 
message bit index, setting the message size index equal to the number of bits in the 

20 message bitstream, and clearing a message encoded flag; 

F) reading a window of samples of the inputted message and 
storing the samples sequentially in a sample buffer; 

G) computing a spectral transform of the samples in the buffer; 

H) obtaining the bit value stored in the map table, wherein if the 
25 bit value is true, then encoding the bit of the message indicated by the message bit 

index into the current sample and incrementing the message bit index, where the 
message bit index equals the number of message bits, and then reperforming step 
A), setting the message encoded flag, and exiting the loop; 

I) computing the inverse spectral of the spectral values stored 
30 in the sample buffer. 
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J) outputting the values in the sample buffer, and if the sample 
encoded flag is set, thai dear the flag and jump bade to step E); 

K) incrementing the convohition index and when the 
convolution index equals the length of the convolution mask in bits resetting the 
convolution index; and 

L) jumping back to step F). 

5. The method of claim 3 wherein the encoding of the message bit into the 
sample in step I indudes encoding a single bit of the sample to match the message 
bit. 

6. The method of daim 4 wherdn the encoding of the message bit into the 
sample in step H indudes altering the sample value such that said sample value falls 
within a prespedfied range of valves rdative to its original value. 

7. A method of decoding information from a sample stream of data, 
comprising the steps of: 

A) obtaining a mask set including: 

(1) a random or ps»ido>random primaiy mask, 

(2) a random or pseudo-random convolution mask, and 

(3) a random or psaido-random start of message ddimitei; 

B) loading a map table, and the mask set into a memory; 

C) resetting a primary mask index and convolution mask index 
and setting a message size integer equal to 0; 

D) clearing a message decoded flag; 

E) setting a state ofthe decode process to SYNCHRONIZED; 

F) . checking the state of the decode process and if the decode 
state is UNSYNCHRONIZED, setting a number of samples to equal 1 and resetting 
the convohition index to 0; otherwise, setting the number of samples to equal S 
(Sil); 
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G) reading the number of samples specified in step F) into a 

8anq>le buffer, 

H) resetting the primaiy mask index, and looping through the 
sample buffer fiom the first sample to the last sample, incrementing the prnmry 

5 mask index each time, and for each sample position, computing the value of a 
mapping function to calculate an ofl&et into the map table; 

I) obtainmg the bit value in the map table, and if the value is true, 
decoding the bit of the message indicated by the message bit index, storing the bit 
into the message buffer at the message bit index, and incrementing the message bit 

10 index; 

J) comparing the decoded bits in the message buffer to the start 
of message deHmiter, and if the number of bits in the message buffer is less than or 
equal to the mimber of bits in the start of message delimiter and the bits match, then 
setting the state of the decode process to SYNCHRONIZED; otherwise setting the 
state of the decode process to UNSYNCHRONIZED; 

K) 'fthe state ofthe decode process is SYNCHRONIZED and 
the number of bits in the message buffer is greater than or equal to the sum ofthe 
number of bits of the start of delimiter and the message size, then setting the state 
ofthe decode process to SYNC-AND-SIZE and copying certain bits fi-om the 
20 message buffer to a message size integer container; 

L) if the state ofthe decode process is SYNC-AND-SIZE and 
the number of bits in the message buffer divided by 8 is greater than or equal to the 
message size, then setting the message decoded flag, outputting the message and 
the message decoded flag and ending the method; 

M) incrementing the convolution index, and if the convolution index 
equals the number of bits in the convolution mask resetting the convolution index; 
and 

N) jumping to step F). 



15 



25 



30 



8. A method of decoding information from sampled data, comprising the steps 
of: 
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A) Obtaining a mask set including 

(1) a random or pseudo*random primary mask» 

(2) a random or pseudo-random convolution mask, and 

(3) a random or pseudo-random start of m^sage 

5 delimitei; 

B) loading a map table, and the mask set into a memory; 

C) resetting a primary mask index and convolution mask index 
and setting a message size integer equal to 0; 

D) clearing a message decoded flag; 

10 E) setting a state ofthe decode process to SYNCHRONIZED; 

F) checking the state of the decode process and if the decode 
state is UNSYNCHRONIZED, setting a number of samples to equal 1 and resetting 
the convolution index to 0; otherwise, setting the number of samples to equal S 
(S>1); 

15 G) reading the number of samples specified in step F) into a 

sample buffer; 

H) computing a spectral transform ofthe samples stored in the 

sample buffer; 

I) resetting the primary mask index and looping through the 
20 sample buffer from the first sample to the last sample, incrementing the primary 

. mask index each time, and for each sample position, computing the value of a 
mapping fiinction by using the bit of the primary mask corresponding to the primary 
mask index and the bit of the convolution masks indicated by the convohition phase 
to calculate an offset into the map table representing the mapping function; 

25 J) obtaining a bit value stored in the map, and if the value is 

true, decoding the bit of the message indicated by the message bit ind«c from the 
current sample, storing the bit into the message buffer at the message bit index, and 
incrementing the message bit ind^ 

K) comparing the decoded bits in the message buffer to the start 

30 of message delinuter, and if the number of bits in the message buffer is less than or 
equal to the number of bits in the start of message delimiter and the bits match, then 



wo 96/42151 



45 



PCT/US9d/10257 



setting the state of the decode process to SYNCHRONIZED; otherwise, setting the 
state of the decode process UNSYNCHRONIZED; 

L) if the state of the decode process is SYNCHRONIZED, and 
the number of bits m the message buffer is greater than or equal to the sum of the 
number of bits of the start of ddimita^ and the message size, then setting the state 
of the decode process to SYNC-AND-SIZE and copying certain bits from the 
message buffer to a message size integer container, 

M) if the state of the decode process is SYNC-AND-SIZE and 
the number of bits in the message buffer divided by 8 is greater than or equal to the 
message aze, then setting the message decoded flag, outputting the message and 
the message decoded flag and rading the method; 

N) incrementing the convolution index, wherein if the 
convolution index equals the number of bits in the convolution mask, then resetting 
the convolution index; and 

O) jumping to step F). 

9. The method of claim 7 wherein the decoding of the message bit from the 
sample in step I includes readmg a single bit of the sample. 

10. The method of claim 7 wherein the decoding of the message bit from the 
sample m step I mcludes mapping a range of sample values onto a particular 
message bit value. 

1 1 . The method of claim 4 wherein the map table is defined such that any index 
of the map table directs the process to encode information. 



12. The method of claim 1 wherein the samples are obtained from a sample 
stream representing digitized sound or music. 
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13. The method of claim 12 wherein the identical encode process is performed 

on two sample streams representing channel A and channel B of digitized stereo 

sound. f 

5 14. The method of claim 12 wherein the sample streams represent channel A 
and channel B of digitized stereo sound and are interleaved before being input as a 
angle sample stream and are separated into two channels upon output. 

15. The method of daim 1 wherein the samples are obtained from a sample 
10 stream representing digitized video. 

16. The method of claim 1 wherein the samples are obtained from a sample 
stream rq)resenting a digitized image. 

*5 17. The apparatus of claim 2, further comprising a tamper-resistant packaging, 
enclosing said apparatus wherein circuitry and information stored therein are 
destroyed if said packaging is opened. 



18. The method of claim 3, fiirdier comprising a pre-encoding step vAich 
customizes the message to be encoded including: calculating over which windows 
in the samples stream a message will be encoded, computing a secure one way hash 
function of the samples in those windows, and placing the resulting hash values in 
the message before the message is encoded; 

wherein the hash calculating step includes: calculating the size of the 
original message plus the size of an added hash value, and pre-processing the 
sample stream for the purpose of calculating hash values of each series of windows 
that will be used to encode the message and creating a modified copy of the 
message containing the hash value such that each message containing a hash value 
matches each window series uniquely. 
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19. The method of clwm 1, wherein an authority for on line distribution of 
content encodes at least one of the following itenis into a sample stream ; 

the title, 
the artist, 
5 the copyright holder, 

the body to which royalties should be paid, and 
general terms for publisher distribution. 

20. The method of claim 19, wherein the authority combines at least one item 
10 with a secure private key signed message from a publisher containing at least one of 

the following pieces of information: 
the title, 

the publisher's identification, 

the terms of distribution, 
15 any consideration paid for the right to distribute the content, 

a brief statement of agreement, and 
the publisher signs and encrypts the combined message using a public 
crypto^stem and encodes the signed and enciypted message into the sample 
stream. 

20 

21 . The method of clahn 20, wherein a publisher obtains the encoded sample 
stream and additionally obtains information form the authority and combines this 
with a message recdved from a consumer, which has been signed using a public key 
cryptosystem and wherein the signed message contains at least one of the following 

25 data 

the content title, 
consumer identification, 
the terms of distribution, 
the consideration paid for the content, 
30 a brief statement of agreement, and 
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the publish^ uses a public key oyptosystem to sign the combined infonnation and 
finally encodes the agned infonnation. 

22. The method of claim 1 , wherein the sample stream is obtmned fi-om at least 
one audio track contained withui a digitized movie, video game software, or other 
software. 

23. The method of claim 1, wherein the sample stream is obtained firom at least 
one digitized movie or still image contained within a video game or other software. 

24. The method of claim 1, wherein encoded information is contained in the 
differences or relationship between samples or groups of samples. 



25. The method of claim 4, wherein the encoding of the message bit into the 
15 sample in step H includes encoding a single bit of the sample to match the message 

bit 

26. The method of claim 3, wherein the encoding of the message bit into the 
sample in step I includes altering the sample value such that said sample value falls 

20 within a prespecified range of valves relative to its original value. 

27. The method of claini 8, wherein the decoding of the message bit in step J 
includes reading a single bit of the sample. 

25 28. The method of claim 8, wherein the decoding of the message bit in step J 
includes mapping a range of supply values onto a particular message bit value. 
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29. The method of claim 3, wherein the map table is defined such that any index 
of the map table directs the process to encode infonnation. 
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30. The method of claim 7. wherdn the map table is defined such that any mdex 
of the map table directs the process to encode mformation. 

31. The method of claim 8, wherdn the map table is defined such that any index 
of the map table directs the process to encode information. 

32. The method of claim 4, further comprising a pre-encoding step which 
customizes the message to be encoded including: calculating over which windows 
in the samples stream a message will be encoded, computing a secure one way hash 
function of the samples in those windows, and placing the resulting hash values in 
the message before the message is encoded; 

wherdn the hash calculating step includes: calculating the size of the 
original message plus the size of an added hash value, and pre-processing the 
sample stream for the purpose of calculating hash values of each series of windows 
that will be used to encode the message and creating a modified copy of the 
message containing the hash value such that each message containing a hash value 
matches each window series uniquely.— 



